Statistical Quasi-Newton: A New Look at Least Change
نویسندگان
چکیده
منابع مشابه
Statistical Quasi-Newton: A New Look at Least Change
A new method for quasi-Newton minimization outperforms BFGS by combining least-change updates of the Hessian with step sizes estimated from a Wishart model of uncertainty. The Hessian update is in the Broyden family but uses a negative parameter, outside the convex range, that is usually regarded as the safe zone for Broyden updates. Although full Newton steps based on this update tend to be to...
متن کاملLeast-change quasi-Newton updates for equality-constrained optimization
January 28, 2000 Abstract. This paper investigates quasi-Newton updates for equality-constrained optimization. Using a least-change argument we derive a class of rank-3 updates to approximations of the one-sided projection of the Hessian of the Lagrangian which keeps the appropriate part symmetric (and possibly positive definite). By imposing the usual assumptions we are able to prove 1-step su...
متن کاملA New Projected Quasi-Newton Approach for the Nonnegative Least Squares Problem
Constrained least squares estimation lies at the heart of many applications in fields as diverse as statistics, psychometrics, signal processing, or even machine learning. Nonnegativity requirements on the model variables are amongst the simplest constraints that arise naturally, and the corresponding least-squares problem is called Nonnegative Least Squares or NNLS. In this paper we present a ...
متن کاملQuasi-Newton Methods: A New Direction
Four decades after their invention, quasiNewton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newton methods can be interpreted as approximations of Bayesian linear regression u...
متن کاملQuasi-newton Methods for Nonlinear Least Squares Focusing on Curvatures
Most existing quasi-Newton methods for nonlinear least squares problems incorporate both linear and nonlinear information in the secant update. These methods exhibit good theoretical properties, but are not especially accurate in practice. The objective of this paper is to propose quasi-Newton methods that only update the nonlinearities. We show two advantages of such updates. First, fast conve...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: SIAM Journal on Optimization
سال: 2008
ISSN: 1052-6234,1095-7189
DOI: 10.1137/040614700